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(54) Title: ADAPTIVE MOTION ESTIMATOR 

(57) Abstract 

A method and apparatus of encoding digital video 
according to the ISO/IEC MPEG standards flSO/IEC 
11172-2 MPEG-1 and ISO/IEC 13818-2 MPEG-2) 
using an adaptive motion estimator. A plurality of global 
motion vectors are derived from the motion vectors of 
a previous picture in a sequence, and the global motion 
vectors are analysed to determine motion characteristics. 
The video encoding is arranged to enable switching 
among different types of local motion estimators based 
on the motion characteristics of the moving pictures 
sequence. This gives rise to a motion estimation 
algorithm that can adaptively change its search range, 
search area and block matching scheme to suit different 
types of moving sequences. 
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Adaptive Motion Estimator 

Field nf the Invention 

The present invention relates to a method and apparatus for encoding moving pictures 
5 sequences. In particular, the present invention relates to a method and apparatus for motion 
estimation and motion compensation in a video signal compression system. 

Background of the Invention 

Methods for encoding moving pictures or video such as the MPEG1, MPEG2, H.261, and 
10 H.263 standards have been developed for efficient data transmission and storage. A detailed 
description of one such encoding method is found in MPEG2 Test Model 5, ISO/IEC 
JTC1/SC29/WG11/N0400, April 1993, and the disclosure of that document is hereby 
expressly incorporated herein by reference. In the described encoding method, an input video 
sequence is organized into a sequence layer, group-of-pictures (GOP), pictures, slices, 
15 macroblocks, and finally block layer. Each picture is coded according to its determined 
picture coding type. The picture coding types used include intra-coded picture (I-picture), 
predictive-coded picture (P-picture), and bi-directionally predictive-coded picture (B-picture). 

Motion estimation/compensation, transform coding, and statistical coding are utilized to 
20 efficiently compress the input video sequence. For example in MPEG2 Test Model 5, each 
picture from the input video sequence is partitioned into rows of smaller and non-overlapping 
macroblocks of picture elements (pixels). Macroblocks in each row may be grouped into one 
or more slices. The compression is performed on each macroblock on a row-by-row basis 
starting from the leftmost macroblock to the rightmost macroblock, and the top row to the 
25 bottom row. 

In the motion estimation/compensation method, motion vectors are detected for each 
macroblock in a picture. The coding mode for a macroblock (e.g. intra-coded, forward- 
predicted, backward-predicted, or interpolated) is decided based on the detected motion 
30 vectors and the determined picture coding type. The utilized motion vectors are differentially 
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coded with variable length codes before outputting. 

A typical motion vector detection process comprises determining, for each macroblock to be 
coded, a search window consisting of pixels from a reference picture and matching pixel 
5 values of the macroblocks to blocks of pixel values obtained from the search window. This 
process is known to be computationally intensive. Particularly, the size of the search window 
has a direct impact to the computation load. 

Many methods of matching the pixel blocks are available, such as an exhaustive search 
10 method which compares every definable block within the search window, a logarithmic search 
method, a hierarchial search, and various other possible derivations. Depending on 
application requirements, a search method may be selected based on its performance in terms 
of accuracy and computation complexity. 

To cater for sequences with large object movements between pictures, methods exist to 
increase the search range without enlarging the search window. These methods typically 
incorporate some form of prediction into the motion vectors, based on certain assumptions, 
to provide greater accuracy motion vectors for picture sequences with large movements 
without a large increase in computation load. One such method is the telescopic search 
method in which the motion vectors of macroblocks from a previously coded or matched 
picture are used to generate a new search window for each current macroblock. The 
telescopic search method comprises the steps of obtaining a motion vector from a co-sited 
macroblock from a closest coded picture; optional scaling of the obtained motion vector 
according to the picture distances between the reference picture, the closest coded picture, and 
the current picture; and defining the search window based on the centre position of the current 
macroblock plus an offset defined by the scaled motion vector. 

Alternate methods of determining search windows are disclosed in United States Patents 
5,473,379 and 5,657,087, for example. The methods disclosed therein comprise the steps 
30 of calculating a global motion vector based on the motion vectors of a previous picture, and 
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offsetting search windows of all macroblocks by the calculated global motion vector. The 
global motion vector may be determined by the mean or the median function, or by the most 
common motion vector of the previous picture; it can be further normalized according to the 
picture distances. The calculated global motion vector may then represent a global 
5 translational motion of objects from one picture to the other. 

There are also hybrid motion estimators which combine both full search and hierarchical 
search to take advantage of the accuracy of full search and wide coverage of hierarchical 
search under a certain hardware limitation. For example, US Patent 5,731,850 discloses a 
10 system in which either full search or hierarchical search is chosen based on the search range 
imposed on various picture types. A full search is chosen if the search range assigned to that 
picture is below a certain threshold, else a hierarchical search is chosen. 

Current arts use a fixed search range and one set of search windows for the various picture 
15 types in encoding a moving sequence, which fails to address the problem of varying motion 
characteristics within a moving sequence. A sequence may consist of segments with different 
characteristics: one segment may consist of slow moving objects with stationary background, 
another may consist of fast moving objects with stationary background, yet another with fast 
moving objects and background, and many other combinations. With such complex motion 
20 characteristics, having a fixed search range for individual picture types is inefficient as it over 
services during the slow moving segments while under servicing fast moving segments. This 
results in non-uniform motion estimator performance and inefficient bit allocation to coding 
the motion vectors. All these factors will lower the general performance of the encoder and 
also result in non-uniform output bitstreams quality. 

25 

Motion estimators of the type disclosed in US Patent 5,731,850 can use a hybrid of full 
search and hierarchical search to take advantage of the accuracy of full search and wide 
coverage of hierarchical search, but the search range is still pre-assigned and does not take 
account of the possible different motion characteristics of a moving sequence. Thus, this 
30 form of motion estimator will not have a good adaptability to moving sequences with large 
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motion variances. The motion estimator therein disclosed is more concerned in offering 
trade-off in accuracy and wide coverage given a certain hardware limitation and a pre- 
assigned search range. 

5 Methods utilising the global motion vector such as disclosed in the aforementioned US Patents 
5,473,379 and 5,657,087 may be used to minimise search window cache size as well as the 
bandwidth requirement from the frame memory while expanding the actual search range. 
These methods fix the offset of the search window for all macroblocks in a picture. 
However, given that only one global motion vector is used for the offset of all search 

10 windows in a picture, the search range expansion works well only with pictures containing 
uniform translational motion. Pictures with zooming, rotational motion, sheering effects and 
pictures with more than one group of translational motions are not well exploited. 

Sunnwry of the invention 
15 In accordance with the present invention, there is provided a method for improved data block 

matching in a moving pictures encoder for encoding a sequence of pictures each comprising 

a plurality of data blocks, comprising the steps of: 

processing at least a picture in the sequence to obtain block motion vectors for the data 

blocks therein with respect to a reference picture; 
20 generating a plurality of global motion vectors for the picture, each global motion 

vector being generated from a plurality of block motion vectors from a respective group of 

related blocks in the picture; 

analysing the global motion vectors to determine a metric representing a distribution 

pattern thereof; 

25 selecting a motion estimation scheme from amongst a plurality of motion estimation 

schemes, for data block matching of at least one subsequent picture in the sequence; and 

performing data block matching of at least one subsequent picture in the sequence 
using the selected motion estimation scheme, said global motion vectors and preselected 
search window characteristics. 

30 
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The present invention also provides a moving pictures encoder for encoding a sequence of 
pictures each comprising a plurality of data blocks, including an adaptive data block matching 
apparatus comprising: 

a global motion estimator coupled to receive block motion vectors for data blocks of 
5 a previously processed picture with respect to a reference picture for generating a plurality 
of global motion vectors for the picture, each global motion vector being generated from a 
plurality of block motion vectors from a respective group of related blocks in the picture; 

a motion characteristics analyser coupled to receive the global motion vectors from 
the global motion estimator for analysing the global motion vectors to determine a metric 
10 representing a distribution pattern thereof; 

a selector coupled to receive the distribution pattern metric from the motion 
characteristics analyser for selecting a motion estimation scheme from amongst a plurality of 
motion estimation schemes, for data block matching of at least one subsequent picture in the 
sequence; and 

15 a plurality of motion estimators controlled by said selector and coupled to receive said 

global motion vectors for performing data block matching of at least one subsequent picture 
in the sequence using the selected motion estimation scheme, said global motion vectors and 
preselected search window characteristics. 

20 The present invention also provides a method for encoding moving pictures data from a 
sequence of moving pictures in which each picture in the sequence is represented by a 
plurality of data blocks corresponding to non-overlapping areas of the picture, the method 
comprising: 

selecting a group of related data blocks from the plurality of data blocks of a picture; 
25 for each data block in the selected group, obtaining a corresponding block motion 

vector from a previously processed picture in the moving pictures sequence; 

determining a plurality of global motion vectors for the selected group, each of the 
global motion vectors being formed from a plurality of the corresponding block motion 
vectors; 

30 analysing the global motion vectors and determining a metric representing a 
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distribution pattern thereof; 

selecting a motion estimator scheme on the basis of the distribution pattern metric, the 
motion estimator scheme being selected from amongst a plurality of motion estimator schemes 
each having a different combination of search strategy and number of global motion vectors; 
5 performing data block-matching with a reference picture using the selected motion 

estimator scheme to generate a block motion vector; and % 

encoding the picture data including the block motion vectors. 

10 The present invention further provides a method for motion estimation for use in encoding 
a picture in a moving pictures sequence wherein data representing the picture in the sequence 
comprises a plurality of data blocks, the method comprising the steps of: 

selecting a group of related data blocks from the plurality of data blocks of the picture; 

for each data block in the selected group, obtaining a corresponding block motion 
15 vector from a previously processed picture in the moving pictures sequence; 

determining a primary global motion vector for the selected group from all of the 
corresponding block motion vectors; 

classifying the block motion vectors from the selected group into a plurality of sub- 
groups;- 

20 determining a plurality of secondary global motion vectors corresponding to the 

respective sub-groups from the block motion vectors classified in the respective sub-groups; 
and 

selecting the primary and/or at least one of the secondary global motion vectors for 
use in defining one or more search windows for each block in the selected group to enable 
25 block matching with a reference picture. 

Embodiments of the present invention aim to provide: 

an improved moving sequence encoding method and apparatus containing a motion 
estimator which can adaptively determine the motion estimation scheme best suited to the 
30 moving sequence based on the motion characteristics; 
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an improved moving sequence encoding method or apparatus containing a motion 
estimator which can adaptively adjust the search range on the fly to suit the motion 
characteristics of the moving sequence; and 

an improved moving sequence encoding method or apparatus containing a global 
5 motion estimator with expanded search range and multiple search windows. 

A preferred form of the present invention utilises a controller to determine which motion 
estimator to use, based on the motion characteristics of the moving sequence to achieve best 
output quality under a certain hardware limitation. The types of motion estimator which can 

10 be utilised are various form of global motion estimators that utilise techniques such as full 
search, hierarchical search, telescopic search and group motion vectors prediction. The term 
"group" in this context refers to a plurality of macroblocks, for example slice(s), row(s), half- 
row etc. The term "motion characteristics" here refers, without limitation, to size of motion 
vectors, distribution pattern of motion vectors, statistical data of motion vectors belonging to 

15 each group and relationships between global motion vectors representing each of these groups. 
With such a scheme, the search range used can be according to the motion characteristics of 
the moving sequence, for example large search range for fast motion and small search range 
for slow moving sequence, this will lead to a better allocation of bits between motion vectors 
and DCT coding. 

20 

An embodiment of the present invention utilises a global motion estimator which detennines 
one or more global motion vectors that best represent one or more groups of MBs in a picture 
to be coded. The global motion estimator may determine the global motion vectors based on 
detected motion vectors from the corresponding group of MBs of a previously processed or 

25 coded picture. Each determined global motion vector provides an offset to all search 
windows for all MBs in the group. The global motion vectors better adapt to motion variation 
within a picture and expand the effective search range of the motion estimator without 
increasing the search window size. Having more than one global motion vector enables cases 
of varying clusters of motion within a group to be covered. Of course, more global motion 

30 vectors per group implies increased computation requirements if a similar search window size 
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is maintained, or a reduction in search window size if the same computational restrictions are 
maintained. 

With the determined set of global motion vectors for a picture, a maximum offset vector can 
5 be computed. The maximum offset vector is preferably the maximum of absolute values of 
the horizontal and vertical components from the set of global motion vectors. Combining 
maximum offset vector and the search window size, a maximum possible motion vector size 
can be determined and therefore the corresponding Variable Length Coding (VLQ tables can 
be selected for coding of motion vectors for the picture. Combining this with the ability to 
10 adaptively change the search range/area according to different motion characteristics, the 
VLC tables selected will be optimised for the types of motion in the picture and this optimises 
the efficiency of the motion vector coding. 

Brief Description of the Drawings 
15 The invention is described in greater detail hereinafter, by way of example only, through 
description of preferred embodiments thereof and with reference to the accompanying 
drawings in which: 

Figure 1 is a functional block diagram of a video encoder with an adaptive motion 
estimator according to one embodiment of the present invention; 
20 Figure 2 is a functional block diagram of an adaptive motion estimator according to 

one embodiment of the present invention; 

Figure 3 illustrates the outcome of performing a vector quantisation on a group of MB 
motion vectors according to one embodiment of the present invention; 

Figure 4 illustrates how multiple search windows are defined by global motion 
25 vectors within a reference picture according to the present invention; 

Figure 5 is a flow-chart illustrating a particular example of how a group of motion 
vectors may be partitioned into sub-groups, according to one embodiment of the present 
invention; and 

Figure 6 is a flow-chart illustrating a particular example of the motion characteristics 
30 extraction, evaluation process for deciding which motion estimator to use, according to one 
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embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

A picture sequence encoder according to a preferred embodiment of the present invention 
5 encodes each input picture by determining its picture coding type (I-, P-, B-picture), obtaining 
MBs from the picture, subjecting MBs of P-picture or B-picture to a motion vector detection 
process (frame and/or field, forward and/or backward) using a global motion estimator, 
performing necessary motion compensation (predicted or interpolated) using the detected 
motion vectors, and subjecting all MBs to a transform coder followed by a statistical coder. 
10 The motion characteristics of past pictures are then collected and input into a controller to 
determine the type of motion estimator to be used for subsequent pictures. 

The global motion estimator is updated with MB motion vectors from the past processed 
pictures by the adaptive motion estimator. The preceding pictures motion vectors are used 
15 to generate one or more global motion vectors for each group of MBs in a new picture to be 
coded based on the type of global motion estimator selected. Generated global motion vectors 
are used to offset search windows of all MBs in the corresponding group of MBs. For the 
case of more than one global motion vector, a comparison at MB level is done and the global 
motion vector that gives the best result is chosen. 

20 

At the end of a picture, a maximum offset vector is determined from all local motion vectors 
of the picture. The maximum offset vector is combined with the maximum search window 
size to select the VLC tables for coding of motion vectors for the picture. 

25 At the end of one or more pictures, a set of motion characteristics is collected from the MB 
vectors supplied by the motion detector, such as size of motion vectors, distribution pattern 
of motion vectors, statistical data of motion vectors belonging to each group of MBs and the 
relationships between global motion vectors of sub-groups with the same group. These are 
used for determining which type of motion estimator is to be used to code the subsequent 

30 pictures(s). The validity period of each decision can be one picture or a plurality of 
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picture(s), after which another decision will be made on another picture or plurality of 
pictures. 

A video encoder with an adaptive motion estimator according to a preferred embodiment of 
5 the present invention is illustrated in block diagram form in Figure 1. In the video encoder 
as shown, a moving picture sequence is input to a frame buffer 101 for recording and 
processing. The moving pictures sequence may be pre-processed in conventional ways before 
being received in the frame buffer 101. 

10 A picture coding type (I, P or B-picture) is determined according to application needs for 
each picture in the input picture sequence. An I-picture is an intra-coded picture used mainly 
for random access or scene update. P-pictures use forward motion predictive coding with 
reference to a previously coded I or P picture (anchor picture), and B-pictures use forward 
and backward motion predictive coding with reference to previously coded I and/or P 

15 pictures. An input picture sequence may be either a field or frame structured sequence 
coming from an interlaced or progressive source. 

Macroblocks containing blocks of pixel values are derived from a picture to be coded from 
the picture data stored in the frame buffer 101 . The MBs are derived on a row by row basis 
20 starting from the leftmost MB to the rightmost MB, and the top row to the bottom row. MBs 
belonging to an I-picture are subjected to a transform coder 105 directly; and MBs belonging 
to a P-picture or B-picture are subjected to an adaptive motion estimator 102. It is also 
possible to subject MBs belonging to an I-picture to the adaptive motion estimator 102 for 
generation of error concealment motion vectors such as defined in the MPEG2 standard. 

25 

All necessary motion vectors, for example the frame and/or field, forward and/or backward, 
and 16x16/16x8/8x8 motion vectors, for each MB are detected by the adaptive motion 
estimator 102 by matching the MB to candidate blocks obtained from one or more search 
windows from a reference picture stored in a frame buffer 103. Different matching methods 
30 such as the exhaustive search method, the logarithmic search method, multi-steps or 
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hierarchical search method, and search windows sizes and numbers may be utilised depending 
on application/implementation needs as well as the type of motion estimator selected within 
the adaptive motion estimator 102. Matching methods may also be implemented in various 
pixel resolutions, for example integer, half pel or quarter pel resolution. The matching 
5 criterion may be based on minimum of absolute errors, square errors, or other suitable 
distortion functions. A detailed description of one form of the adaptive motion estimator 102 
is presented hereinbelow in connection with Figure 2. 

A motion compensation processor 104 is coupled to the adaptive motion estimator 102. In 
10 motion compensation process 104, a MB coding mode, such as intra-coding, frame/field 
forward/ backward prediction, or frame/field interpolation, is first decided for each MB based 
on the detected motion vectors from the adaptive motion estimator 102. Necessary prediction 
errors are then generated based on the decided MB coding mode. An example of a MB 
coding mode decision may be found in the MPEG2 Test Model 5. 

15 

Macroblocks resulting from the motion compensation process 104 are the subjected to a 
transform coder 105 which exploits correlation within each MB and also its psycho-visual 
effects. Examples of transform coders may be found in the aforementioned MPEG1, 
MPEG2, H.261 and H.263 standards. An embodiment of a transform coder 105 according 

20 to the MPEG2 Test Model 5 consists of a DCT, quantiser rate controller with adaptive 
quantisation, inverse quantiser, and inverse DCT. The transformed and quantised coefficients 
are inverse quantised and inverse transformed by the transform coder 105 to produce 
reconstructed MBs which are passed to the frame buffer 103 for future reference. Necessary 
inverse of motion compensation is also provided to each reconstructed MB by the motion 

25 compensation process 104. Reference pictures formed by the reconstructed MBs are used in 
the adaptive motion estimator 102 and motion compensation process 104. In some 
applications or coding instances, it is also possible to take input pictures directly as reference 
pictures for the adaptive motion estimator 102. 



30 A statistical coder 106 is coupled to the transform coder 105, which exploits statistical 
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redundancies in the received data, and multiplexes the results to produce the final compressed 
output bitstreams. As an example in the MPEG2 Test Model 5, the statistical coder 106 
provides the zig-zag scanning and run-length encoding of the transformed and quantised 
coefficients, differential coding of the utilised motion vectors, and multiplexing of all results 
5 and necessary side information (e.g.. sequence/GOP/picture/slice/MB layer header 
information, picture coding types, MB coding modes, etc.). The statistical coder 106 utilises 
variable length codes (VLC) from VLC tables. 

A functional block diagram of one form of the adaptive motion estimator 102 is illustrated 
10 in Figure 2. The adaptive motion estimator of this embodiment has two inputs 208, 209 and 
two outputs 210, 211. Current picture data blocks are supplied to a selector 201 of the 
adaptive motion estimator from the frame buffer 101 at input 209. Reference picture data is 
provided from the reference picture frame buffer 103 at input 209 which is coupled to a 
plurality of motion estimators 204, 205, 206 and 207. The selector 201 also receives input 
15 from a motion characteristics analyser 202. The motion estimators 204, 205, 206, 207 are 
controlled by the selector so that a selected one of them produces an output to the motion 
vector output 210, which is provided to the motion compensation processor 104 (Figure 1). 
The output motion vector is also fed back to a global motion estimator 203 and to the motion 
characteristics analyser 202. The global motion estimator 203 is coupled to provide data to 
20 the motion characteristics analyser and the motion estimators 204, 205, 206, 207. The global 
motion estimator 203 also produces a maximum search range output which is provided to the 
statistical coder 106 (Figure 1). 

The global motion estimator 203 determines one or more global motion vectors for each row 
25 of MBs from an input picture. In this particular embodiment, three global motion vectors are 
computed: one representing the average motion vector of the whole row, the other two global 
motion vectors representing two sub-groups of MBs from the row differentiated by some 
motion characteristics. For this embodiment, the two sub-groups are obtained by performing 
two-level vector quantisation on the MB motion vectors in the row. An example of a two- 
30 level quantisation process is illustrated in flow diagram form in Figure 5. 
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Referring to Figure 5 beginning at step 501, firstly a global motion vector gmv is computed 
by averaging the MB motion vectors in the current row (step 502). The global motion vector 
gmv is then perturbed at step 503 to obtain separate initial values for secondary motion 
vectors gmvl and gmv2. The vector quantisation algorithm then enters, an iteration loop 

5 beginning at step 504 whereby the distance of each of the MB motion vectors from gmvl and 
gmvl are being compared to separate the MB motion vectors into two groups, Groupl and 
Group!. The secondary motion vectors gmvl and gmvl are then re-computed from the 
motion vectors in the respective new groupl and groupl. At step 504 a test is applied to 
determine if a preset maximum number of iterations has yet been reached. If the maximum 

10 number of iterations has been reached then the procedure ends at step 511 with the primary 
and secondary global motion vectors which have been computed. If the iteration number limit 
has not been reached, then the distances between each of the respective macroblock motion 
vectors and the secondary global motion vectors gmvl and gmvl are computed at step 505. 
At steps 506, 507 and 508, each of the MB motion vectors is determined to be closer to either 

15 gmvl or gmvl. If the MB motion vector is closer to the current value of gmvl then gmvl 
then it is assigned to Groupl, otherwise it is assigned to Groupl. In the next step (509), the 
value of gmvl is temporarily preserved in a register prevjgmvl and the value of gmvl is 
preserved in prev jgmvl, following which values for the gmvl and gmvl are recomputed 
according to the MB motion vectors allocated to the respectively to Groupl and Groupl. The 

20 vector quantisation procedure then carries out another test at step 510 to determine whether 
to terminate the iteration loop. The test at step 510 involves a comparison of prevjgmvl with 
recomputed gmvl and prev jgmvl with recomputed gmvl. If the previous and recomputed 
values of the secondary global motion vectors are within a small preset tolerance (i.e. the 
iteration loop leaves gmvl and gmv! essentially unchanged) then the procedure ends at step 

25 511 with the computed global motion vectors gmv, gmvl and gmvl. Thus, the process of 
distance comparison and group reassignment continues until the value of gmvl and gmvl 
remains unchanged, or the maximum number of iterations allowed is reached. 

An example of a distribution of MB motion vectors is illustrated in Figure 3, for the case of 
30 eight MB motion vectors in a row. In this Figure, plain circles represent the placement of 
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individual macroblock motion vectors in the row. The primary global motion vector which 
constitutes the combination of all of the individual MB motion vectors is indicated by the 
symbol at 301. The vector quantisation procedure determines two groupings of MB motion 
vectors, the first group (Group 1) comprising the motion vectors labelled as 304 and the 

5 second group (Group!) being the motion vectors labelled 305. The secondary global motion 
vectors gmvl and gmvl are respective averages of the motion vectors from Groupl and 
Group!, and are indicated in the Figure at 302 and 303. This Figure graphically illustrates 
the relationships between the eight MB vectors, the global motion vector 301 representing the 
whole row (gmv), and the two secondary global motion vectors 302 and 303 representing the 

10 two sub-groups (gmvl and gmv2) are as shown. Note that as an alternative to using vector 
average, other functions may be used to represent each of the groups, such as a median 
function, most common vector, etc. 

The calculated global motion vectors are used in subsequent picture(s) to offset the search 
15 window(s). This is diagrammatically illustrated in Figure 4 for the case of two global motion 
vectors per row. The first search window for a current MB to be coded is determined from 
a reference picture 401 by first determining the position of a co-sited macroblock 402 on the 
reference picture 401. The position of the macroblock 402 is offset according to the first 
global motion vector 403 generated for the MB row to which the current MB belongs. This 
20 results in a new offset MB position 404 as illustrated. Then, the horizontal and vertical 
search ranges are included, centred around the offset MB position 404, to form the search 
window 405 for the current MB. The search window 405 is used for motion vector detection 
for the current MB. The same procedure is carried out for the second global motion vector 
406 to obtain the offset MB 407 and search window 408. The two best macroblock matching 
25 predictions obtained from the two search windows are compared and the one giving the best 
prediction is chosen, with the corresponding MB motion vector assigned to the current MB. 
If only one global motion vector is used (depending on the type of motion estimator chosen), 
then the procedure illustrated previously for the first search window is carried out. 



30 As mentioned earlier, different matching methods such as the exhaustive search method, the 
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logarithmic search method, multi-step or hierarchical search method, and search windows in 
different sizes and numbers may be utilised. Referring again to Figure 2, in this embodiment 
four different motion estimation schemes are considered, labelled FS_GMV1, FS_GMV2 t 
HSJJMV1, and HSJZMV2 in the motion estimator blocks 204, 205 , 206 and 207 

5 respectively. All of the motion estimators 204, 205, 206, 207 are coupled to the global 
motion estimator 203 to obtain global motion vector(s) to offset the search window, but the 
motion estimators differ in terms the type of block-matching technique employed and number 
of global motion vectors used. For example, FSJJMVl uses one global motion vector and 
an exhaustive search block-matching technique. This technique gives the best accuracy 

10 among the three, but gives the smallest search range/area given the same amount of 
computational processing. FS_GMV2 uses two global motion vector and an exhaustive search 
blocking matching technique. This method is useful when there are two groups of motion 
within the pictures; it gives more accurate results than HSJJMV2 but requires much more 
computation (e.g. more hardware) for a meaningful and effective search range. HS_GMV1 

15 uses one global motion vector but a hierarchical search block-matching technique followed 
by multi-step refinements. This gives a good trade off between accuracy and area of 
coverage. Therefore the search range/area can be a few times larger than that of FSJ3MV1 
(given the same hardware) depending on the amount of sub-sampling done on the picture. 
HS GMV2 uses two global motion vectors and a hierarchical search block-matching plus 

20 multi-step refinement technique. This method gives better performance when there are 
significant difference in foreground/background motion. 

The selector 201, together with the motion characteristics analyser 202 are responsible for 
deciding which of the schemes is to be used, at picture level, based on the motion 

25 characteristics of past pictures. The motion characteristics analyser 202 is fed with motion 
vector information from the global motion estimator 203 and the output of one of the motion 
estimators 204 to 207 (depending on which one was selected for the current picture). From 
this information, some metrics representing the distribution pattern of motion vectors in the 
picture are extracted, such as the distance between each sub-group of motion vectors, the 

30 magnitude of group/sub-group global motion vectors, etc. These metrics are passed to the 
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selector 201 for evaluation after which a decision will be made on which motion estimator to 
use for subsequent picture(s). In this exemplary embodiment, only motion estimators 
FSJJMV1 (204), HSJGMV1 (206) and HSJGMV2 (207) are used. The FS_GMV2 motion 
estimator 205 is excluded in this case to keep the hardware size small, it can be included (and 
5 will give very good results) if hardware is expanded sufficiently for the FSJJMV2 motion 
estimator to have a meaningful and effective search range. 

Since correlation of the current picture and pictures in its vicinity is high in the absence of 
scene changes, the type of motion estimation scheme selected is often suitable for pictures in 
10 the vicinity of the current picture. For this particular embodiment, the assignment scheme 
is such that if one of the schemes, say FSJ3MV1, is associated with a particular picture (say 
picture N), then those pictures that used global motion vectors derived from picture N will 
also use FSJJMV1 for the motion vector detection process in adaptive motion estimator 102. 

15 One example of the motion characteristics analysis, evaluation and motion estimator selection 
process used in this embodiment is illustrated in flow diagram form in Figure 6, beginning 
with step 601. The distances of the MB motion vectors from their corresponding global 
motion vectors are first computed and their average value, gmvjrar, obtained for the picture 
(step 602). The value of gmv_var is a measure of compactness of the distribution of motion 

20 vectors. Note that other metrics that measure the compactness of a group of values can also 
be used, for example the variance of MB motion vectors about their global motion vector. 
The value of gmv_yar is compared to a certain preselected threshold, threshold!, (step 603) 
and if the gmv_yar is smaller than thresholdl then the motion estimator FSJGMVl scheme 
is selected (step 604). If the value of gmvjar is determined to be larger than thresholdl at 

25 step 603 then another metric measuring how far apart two distinct groups of MB vectors in 
a row is computed (step 605). This basically divides the MB motion vectors in a row into 
two sub-groups using the two-level vector quantisation and measuring the distance between 
the two global motion vectors representing the groups. A count of how many rows in the 
picture contain distinctly far apart sub-groups of motion vectors is obtained and compared to 

30 another predetermined threshold, threshold^, at step 606. The result of this comparison, 
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together with another comparison of gmv_yar against a second threshold, threshold!, (step 
607) determines whether the strategy HSJGMV1 (step 608) or HSJGMV2 (step 609) will be 
selected. 

5 It will be appreciated from the foregoing detailed description that embodiments of the present 
invention provide improved methods and apparatus for motion vector detection in a video data 
encoder. A motion estimator controller is used to select from among a set of motion 
estimators the most suitable one for a segment consisting of one or more pictures. Selection 
is based on the motion characteristics of past pictures. The controller is therefore able to 

10 adaptively choose the motion estimator that gives the best trade off in terms of accuracy and 
area of coverage, given a certain fixed hardware complexity. All of these features enable 
increases in the performance of the encoder, and enable it to provide high accuracy motion 
estimation for slow moving sequences while still having the ability to cover fast moving 
sequences that require large search range/areas. Embodiments of the invention also allow 

15 adaption of the VLC tables to the different search range selected, resulting in a more efficient 
bit allocation to the coding of motion vectors. 

Embodiments of the present invention also reduce the computational complexity and improves 
the performance of the encoder by using a global motion estimator with multiple search 
20 windows. This enables better prediction of pictures containing not just translational, but also 
zooming, sheer and multiple clusters of different foreground/background motion. With a 
fixed global motion vector, the hardware (e.g. search window cache) required for a large 
search range/area implementation is greatly reduced. 

25 It will be readily understood by those skilled in the art that the invention described herein can 
be practically implemented in a number of different ways. For example, the principles of the 
invention can be incorporated in an integrated circuit for encoding/decoding video data, in 
which case the functions of the invention may be embodied in the circuit design, firmware, 
microcode or the like. The invention may just as easily be implemented on general purpose 

30 data processing apparatus, in which case the functions of the invention may be embodied in 
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a sequence of computer software instructions. 

The foregoing detailed description of the preferred embodiments has been presented by way 
of example only, and is not intended to be considered limiting to the scope of the present 
5 invention which is defined in the claims appended hereto. 
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Claims: 

1. A method for motion estimation for use in encoding a picture in a moving pictures 
. sequence wherein data representing the picture in the sequence comprises a plurality of data 
5 blocks, the method comprising the steps of: 

selecting a group of related data blocks from the plurality of data blocks of the picture; 
for each data block in the selected group, obtaining a corresponding block motion 
vector from a previously processed picture in the moving pictures sequence; 

determining a primary global motion vector for the selected group from all of the 
10 corresponding block motion vectors; 

classifying the block motion vectors from the selected group into a plurality of sub- 
groups; 

determining a plurality of secondary global motion vectors corresponding to the 
respective sub-groups from the block motion vectors classified in the respective sub-groups; 
15 and 

selecting the primary and/or at least one of the secondary global motion vectors for 
use in defining one or more search windows for each block in the selected group to enable 
block matching with a reference picture. 

20 2. A method as claimed in claim 1, wherein the block motion vectors from the selected 
group are classified into the sub-groups according to spatial clustering of the block motion 
vectors. 

3. A method as claimed in claim 1 or 2, wherein the primary and secondary global 
25 motion vectors are computed from an average of the block motion vectors within the 

respective corresponding group or sub-group. 

4. A method as claimed in claim 1, including determining a match between each block 
in the selected group and a matching-block in the one or more search windows for that block 

30 in the reference picture, and determining a computed motion vector between each block in 
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the selected group and its matching block. 

5. A method as claimed in claim 4, including storing the computed motion vector for 
each block for use as a block motion vector for processing subsequent pictures in the moving 

5 pictures sequence. 

6. A method as claimed in claim 5, including generating a computed motion vector for 
each block in the picture and determining a maximum offset vector from the largest of the 
computed motion vectors, and using the maximum offset vector to select a variable length 

10 coding level for the computed motion vectors for the picture. 

7. A method as claimed in claim 1, including selecting and performing one of a plurality 
of motion estimation and search schemes based on selected characteristics of the primary and 
secondary global motion vectors, the plurality of motion estimation and search schemes 

15 employing various combinations of the global motion vectors and matching-block search 
window schemes. 

8. A method for encoding moving pictures data from a sequence of moving pictures in 
which each picture in the sequence is represented by a plurality of data blocks corresponding 
to non-overlapping areas of the picture, the method comprising: 

selecting a group of related data blocks from the plurality of data blocks of a picture; 
for each data block in the selected group, obtaining a corresponding block motion 
vector from a previously processed picture in the moving pictures sequence; 

determining a plurality of global motion vectors for the selected group, each of the 
global motion vectors being formed from a plurality of the corresponding block motion 
vectors; 

analysing the global motion vectors and determining a metric representing a 
distribution pattern thereof; 

selecting a motion estimator scheme on the basis of the distribution pattern metric, the 
motion estimator scheme being selected from amongst a plurality of motion estimator schemes 
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each having a different combination of search strategy and number of global motion vectors; 

performing data block-matching with a reference picture using the selected motion 
estimator scheme to generate a block motion vector; and 

encoding the picture data including the block motion vectors. 

5 

9. A method as claimed in claim 8 including a step of determining, at the end of 
processing of a particular picture, a maximum search range using the all of the global motion 
vectors determined for the picture, and selecting a variable length coding scheme in 
accordance with the maximum search range for encoding the block motion vectors for the 

10 picture. 

10. A method for improved data block matching in a moving pictures encoder for 
encoding a sequence of pictures each comprising a plurality of data blocks, comprising the 
steps of: 

1 5 processing at least a picture in the sequence to obtain block motion vectors for the data 

blocks therein with respect to a reference picture; 

generating a plurality of global motion vectors for the picture, each global motion 

vector being generated from a plurality of block motion vectors from a respective group of 

related blocks in the picture; 
20 analysing the global motion vectors to determine a metric representing a distribution 

pattern thereof; 

selecting a motion estimation scheme from amongst a plurality of motion estimation 
schemes, for data block matching of at least one subsequent picture in the sequence; and 

performing data block matching of at least one subsequent picture in the sequence 
25 using the selected motion estimation scheme, said global motion vectors and preselected 
search window characteristics. 

11. A moving pictures encoder for encoding a sequence of pictures each comprising a 
plurality of data blocks, including an adaptive data block matching apparatus comprising: 

30 a global motion estimator coupled to receive block motion vectors for data blocks of 
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a previously processed picture with respect to a reference picture for generating a plurality 
of global motion vectors for the picture, each global motion vector being generated from a 
plurality of block motion vectors from a respective group of related blocks in the picture; 

a motion characteristics analyser coupled to receive the global motion vectors from 
5 the global motion estimator for analysing the global motion vectors to determine a metric 
representing a distribution pattern thereof; 

a selector coupled to receive the distribution pattern metric from the motion 
characteristics analyser for selecting a motion estimation scheme from amongst a plurality of 
motion estimation schemes, for data block matching of at least one subsequent picture in the 
10 sequence; and 

a plurality of motion estimators controlled by said selector and coupled to receive said 
global motion vectors for performing data block matching of at least one subsequent picture 
in the sequence using the selected motion estimation scheme, said global motion vectors and 
preselected search window characteristics. 

15 

12. A moving pictures encoder as claimed in claim 11, wherein the global motion 
estimator includes means for determining a maximum search range on the basis of the global 
motion vectors. 

20 13. A moving pictures encoder as claimed in claim 12, including a statistical coder 
employing variable length codes, and wherein the statistical coder is coupled to the global 
motion estimator to receive said maximum search range, and wherein the statistical coder 
selects a variable length coding scheme for block motion vectors of a picture on the basis of 
said maximum search range. 



25 



WO 00/70879 



1/6 



PCT/SG99/00041 




U 

64 



If 



WO 00/70879 



2/6 



PCT/SG99/00041 



208 



6- 



Reference 
Pictures 



201 

Input l 

Block i J_ 

Data r— " 



n 



Selector 



209 



202 



Motion 
Characteristics 
Analyser 




Single GMV 
Full Search 
Motion Estimator 
(FSJ3MV1) 



204 



rz 



205 



Double GMV 
Full Search 
Motion Estimator 
(FSJ3MV2) 



210 



206 



Single GMV 
Hierarchical Search 
Motion Estimator 

(HSJ3MV1) 



Output 
Motion 
Vector 



207 



Double GMV 
Hierarchical Search 
Motion Estimator 
(HS_GMV2) 



Maximum 
Search Range 



Figure 2 




Figure 3 



WO 00/70879 



4/6 



PCT/SG99/00041 




WO 00/70879 



5/6 



PCT/SG99/00041 



Compute global 
motion vector for 
whole row, gmv 



501 



502 



Perturbation of gmv 
to obtain 
gmvl and gmv2 



503 




Compute distance of 
MB motion vectors from 
gmvl and gmv2 for current row 



506 




504 



505 



Assigned to 
Group 1 



Assigned to 
Group2 



508 



prev _gmvl = gmvl 
prev Jgmv2 = gmv2 
Recompute gmvl and gmv2 
for current grouping 



IT 

507 




509 



511 



Stop 



3 



Figure 5 



WO 00/70879 



PCT/SG99/00041 



6/6 



607 



C Start J— r 



601 



Compute variance of 
MB motion vectors 
distribution, gmvjvar, 
for one picture 



602 




Compute number of rows, 
SliceVQjCnt, within a 
picture, that contain two 
distinct groups of motion 
vectors 



605 



606 




609 Y 




HS GMV2mode 



HS GMV1 mode 



Figure 6 



INTERNATIONAL SEARCH REPORT 



Inter nal Application No 

PCT/SG 99/00041 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 H04N7/36 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 H04N 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the International search (name of data base and. where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 4 Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



ZIEGLER M: "HIERARCHICAL MOTION 
ESTIMATION USING THE PHASE CORRELATION 
METHOD IN 140 MBIT/S HDTV-CODING" 
PROCEEDINGS OF THE INTERNATIONAL WORKSHOP 
ON HDTV, NL, AMSTERDAM, ELSEVIER, 
vol. WORKSHOP 3, page 131-137 XP000215234 
paragraph '03.2! 

HANG H -M ET AL: "MOTION ESTIMATION FOR 

VIDEO CODING STANDARDS" 

JOURNAL OF VLSI SIGNAL 

PROCESSING, NL.KLUWER ACADEMIC PUBLISHERS, 

DORDRECHT, 

vol. 17, no. 2/03, 1997, page 113-136 
XP000724575 

ISSN: 0922-5773 
page 126, right-hand column, line 10 -page 
127, left-hand column, line 7 

-/-- 



1-13 



1-13 



Further documents are listed in the continuation of box C. 



jy I Patent family members are listed in annex. 



- Special categories of cited documents : 

"A" document defining the general state of the art which is not 
considered to be of particular relevance 

"E" earlier document but published on or after the international 
filing date 

"L" document which may throw doubts on priority claim(s)or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

"P" document published prior to the international filing date but 
later than the priority date claimed 



T later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
Involve an inventive step when the document is taken alone 

"V" document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the an. 

document member of the same patent family 



Date of the actual completion of the international search 



17 January 2000 



Oate of mailing of the international search report 



24/01/2000 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL - 2280 HV Rijswijk 
Tel. (+31-70) 340-2040. Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 



Authorized officer 



Berbain, F 



Foim PCT/ISA/210 Isecond sheet) (July 1992) 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



inter nal Application No 

PCT/SG 99/00041 



C.(Contlnuotk)n) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category » Citation ot document, with indication,whore appropriate, of the relevant passages 



Relevant to claim No. 



JP 10 191347 A (CASIO COMPUT CO LTD) 
21 July 1998 (1998-07-21) 
abstract 

EP 0 831 642 A (NIPPON ELECTRIC CO) 
25 March 1998 (1998-03-25) 
abstract 

page 4, line 7 - line 19 

WO 95 30310 A (MOTOROLA INC) 
9 November 1995 (1995-11-09) 
abstract 

page 8, line 4 - line 19 

EP 0 468 279 A (ANT NACHRICHTENTECH) 
29 January 1992 (1992-01-29) 
abstract 
figure 1 

W0 98 43434 A (PANDEL JUERGEN ;SALAI 
ALBERT (DE); SIEMENS AG (DE)) 
1 October 1998 (1998-10-01) 
abstract 
figure 4 



1-13 



1-13 



1-13 



1-13 



1-13 



Form PCT/ISA/210 (continuation ol second sheet) (Jury 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

..formation on patent family member* 



Inter! lal Application No 

PCT/SG 99/00041 



Patent document 
cited In search report 



Publication 
date 



Patent family 
member(s) 



Publication 
date 



JP 10191347 



21-07-1998 



NONE 



EP 0831642 A 25-03-1998 



JP 10098730 A 



WO 9530310 A 09-11-1995 



US 
AU 
AU 
CN 
EP 
JP 



5537155 A 
688893 B 
2273595 
1128097 
0711488 
8512189 



A 
A 
A 
T 



EP 0468279 A 29-01-1992 



DE 
AT 
DE 



4023449 C 
129112 T 
59106655 0 



14-04-1998 



16- 07-1996 
19-03-1998 
29-11-1995 
31-07-1996 
15-05-1996 

17- 12-1996 



W0 9843434 A 01-10-1998 



DE 
EP 



19712785 C 
0970586 A 



23-01-1992 

15- 10-1995 

16- 11-1995 

16-07-1998 
12-01-2000 



Four) PCTVISA210 (patent tamiy annex) (July 1992) 



